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Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )S Responsive to communication(s) filed on 22 March 2004 . 
2a)D This action is FINAL. 2b)S This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayie, 1935 CD. 1 1 , 453 O.G. 21 3. 

Disposition of Claims 

4) |EI Claim(s) 1-3,8-1 3.1 6-20 and 25-36 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) I3 Claim(s) 1-3.8-13.16-20 and 25-36 is/are rejected. 

7) Q Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)^ The drawing(s) filed on 27 October 2003 is/are: a)S accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1 .121(d). 
1 1 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-1 52. 

Priority under 35 U.S.C. §119 
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a)D All b)D Some * c)D None of: 

1 0 Certified copies of the priority documents have been received. 

2.Q Certified copies of the priority documents have been received in Application No. . 



3.Q Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 



Allowable Subject Matter 



1. The indicated allowability of claims 15 and 16 is withdrawn in view of the reference(s) 
to Smith et al. 6,128,003, cited in Paper No. 8 and not used in any prior rejections. 
Rejections based on the newly used reference(s) follow. 



2. Claim 26 is objected to because of the following informalities: There are 3 periods at 
the end of the claim. Appropriate correction is required. 

3. Claim 34 is objected to under 37 CFR 1 .75(c), as being of improper dependent form. 
Applicant is required to cancel the claim(s), or amend the claim(s) to place the claim(s) 
in proper dependent form. Claim 34 recites the same limitations as claim 26, both of 
which are dependent on claim 1 . Thus, claim 34 is a duplicate claim. 

Claim Rejections - 35 USC §112 

4. The following is a quotation of the first paragraph of 35 U.S.C. 1 1 2: 

The specification shall contain a written description of the invention, and of the manner and process of 
making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the 
art to which it pertains, or with which it is most nearly connected, to make and use the same and shall 
set forth the best mode contemplated by the inventor of carrying out his invention. 

5. Claims 27 and 33 are rejected under 35 U.S.C. 112, first paragraph, as failing to 
comply with the written description requirement. The claim(s) contains subject matter 
which was not described in the specification in such a way as to reasonably convey to 
one skilled in the relevant art that the inventor(s), at the time the application was filed, 
had possession of the claimed invention. As to claim 27, it recites a method which 
comprises "determining the duration of the gesture", which is not found in the claims or 



Claim Objections 
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specification as originally filed. Further, also not found in the specification or claims as 
originally filed is a step of "correlating the duration of the gesture to a intensity and scale 
in which the command is executed." As to claim 33, it recites a method wherein 
"repetition of a command increases the intensity of the corresponding command 
response", which is not found in the specification or claims as originally filed. Since 
these limitations were not disclosed or described in the originally filed specification, they 
constitute new matter and should be deleted. 

6. The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

7. Claims 1-3, 8-13,16-20 and 25-36 are rejected under 35 U.S.C. 112, second 
paragraph, as being indefinite for failing to particularly point out and distinctly claim the 
subject matter which applicant regards as the invention. 

8. Claims 1 and 18 recite the limitation "the trajectory" in line 13 of claim 1 and in line 17 
of claim 18. There is insufficient antecedent basis for this limitation in the claim. 

9. Claim 8 recites the limitation "the step of translating the gesture" in lines 1-2 and the 
limitation "the trajectory" in line 4 and line 6. There is insufficient antecedent basis for 
these limitations in the claim. 

10. Claim 9 recites the limitation "the translation" in lines 10-1 1 and again in line 12. 
There is insufficient antecedent basis for this limitation in the claim. Note that the 
amended claim 9 has "a first translation" in line 5 and "a second translation" in line 6. 





Application/Control Number: 10/001,553 
Art Unit: 2673 



Page 4 



Claim Rejections - 35 USC § 102 



11. The text of those sections of Title 35, U.S. Code not included in this action can be 
found in a prior Office action. 

12. Claims 1-3, 8, 18-20 and 25 are rejected under 35 U.S.C. 102(e) as being 
anticipated by Qiao 6,075,895. 

13. In regard to claims 1 and 18, see previous rejections of claims 1 and 18 in Paper 
Nos. 4 and 6. Qiao further discloses identifying a gesture according to the trajectory of 
the object. See column 6, lines 58-62, disclosing, "After the pre-processor 152 removes 
(step 181) the background image to generate the player's image, the template matcher 
154 (step 183) the player's image to a number of templates to generate template 
outputs". Further see column 8, lines 61-62, disclosing, "the post processor 156 
analyzes those outputs to identify the specific pre-defined gesture that corresponds" 
and lines 65-67, disclosing "the post-processor tracks (step 377) the player's body 
position". Such tracking through templates of body positions is understood to follow a 
trajectory, as best understood. 

Qiao further discloses determining if the gesture corresponds to a valid command, 
and if the gesture corresponds to a valid command, executing the command. See 
column 11, lines 11-16, disclosing, "Other approaches are applicable, such as a rule- 
based system... When an input is received, the ruled-based system compares it with its 
rules to generate the appropriate output." Thus, Qiao determines if the gesture 
corresponds to a valid command by comparing an input with rules and if the gesture 
corresponds to a valid command, the command, or "appropriate output" is executed. 
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Further, see an example of these steps beginning on line 61 of column 1 1 to column 12, 
line 12. 

Further in regard to claim 1 , Qiao automatically controls a device, as best 
understood, because the gestures of a user automatically control the gesture 
recognizing device, in that the device receives the gestures and maps them, classifies 
them, etc., accordingly without further user control. Thus, the gesture recognizing 
device performs many functions automatically. 

14. In regard to claims 2, 3, 8, 19, 20 and 25, see previous rejections of claims 2 and 3 
in Paper Nos. 4 and 6. 

Claim Rejections - 35 USC § 103 

15. The text of those sections of Title 35, U.S. Code not included in this action can be 
found in a prior Office action. 

16. Claims 9, 10, 13, 16 and 17 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Iwamura 6,501 ,515 B1 in view of Prasad et al. 5,680,481 , further in 
view of Smith et al. 6,128,003. 

17. In regard to claim 9, see Paper No. 4, 102(e) rejection of claim 9 and Paper No. 6 
103(a) rejection of claim 14. Iwamura further discloses a vertical movement output 
pattern, a horizontal output pattern and an unknown output pattern. See column 5, lines 
18-28, disclosing, "If the motion detector circuit 15 loses track of the hand 20 [unknown 
output pattern], the motion detector circuit 15 informs the CPU 16 to cause the CRT 9 to 
display the message "Move your hand right". The user 18 follows the message. Then 
the motion detector circuit 15 causes the CPU to control the CRT 9 to display 
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another message "Move your hand upward." The user 18 follows the message again. 
If the motion detector circuit 15 captures the image that moves right first [horizontal 
output pattern and upward next [vertical output pattern], then the motion detector circuit 
15 re-captures and locks on the hand image again." 

Iwamura in view of Prasad further disclose comparing the reference point to a centroid 
upon determining the translation to be a vertical or a horizontal translation. Note, in 
Iwamura, that "the motion detector circuit 15 re-captures and locks on the hand image 
again", upon determining the translation to be a horizontal then vertical translation, as 
disclosed in the previous paragraph. Also in Iwamura, when the motion detector locks 
on the hand image, the hand's motion is then tracked. See column 4, lines 49-52, 
disclosing, "The motion detector circuit 15 recognizes and locks the hand 20 as follows. 
The motion detector circuit calculates a motion vector of each macro block between 
two adjacent frames." Thus, the reference point is compared to an adjacent frame. 
See Paper No. Paper No. 6 103(a) rejection of claim 14, in which Prasad teaches the 
use of a centroid. 

Further in regard to claim 9 and in regard to claim 16, Iwamura in view of Prasad 
does not disclose testing an input pattern upon determining the translation to be an 
unknown translation, wherein the step of testing an input pattern further comprises the 
steps of detecting a circular movement, wherein an angle between vector CtCt-1 and 
vector Ct-1Ct-2 is determined as the feature vector. 

Smith et al. discloses an invention in which an input pattern is tested upon 
determining the translation to be an unknown translation, wherein the step of testing an 
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input pattern further comprises the steps of detecting a circular movement. See column 
10, lines 17-20, disclosing, "Models of different hand gestures are built so that in real 
time, the unknown rotational vector can be compared to all models, the correct hand 
shape being the one with the smallest distance." Smith further discloses that an angle 
between vector CtCt-1 and vector Ct-1 Ct-2 is determined as the feature vector. See 
figure 7 and column 8, lines 16-44. 

Smith further teaches in column 2, lines 64-67 that his invention provides "a hand 
gesture recognition system and method which can be performed effectively with 
relatively low-performance processing circuits." Smith further teaches in column 3, lines 
10-12, "by basing the rotational vectors upon real-valued centroids, there is a smooth 
transition from one image to the next and noise is dramatically reduced." 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to modify the invention of Iwamura in view of Smith by testing an 
input pattern upon determining the translation to be an unknown translation, wherein the 
step of testing an input pattern further comprises the steps of detecting a circular 
movement, wherein an angle between vector CtCt-1 and vector Ct-1 Ct-2 is determined 
as the feature vector, as in the invention of Smith. One would have been motivated to 
make such a change based on the teaching of Smith that such a method provides "a 
hand gesture recognition system and method which can be performed effectively with 
relatively low-performance processing circuits." 

18. In regard to claims 10, 13 and 17, see Paper No. 4, 102(e) rejection of claims 10, 13 
and 17. 



Application/Control Number: 10/001 ,553 Page 8 

Art Unit: 2673 

19. Claims 1 1 and 12 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Iwamura 6,501,515 B1 in view of Prasad et al. 5,680,481, further in view of Smith et al. 
6,128,003 and further in view of Jeannin 6,587,574 B1. 

20. In regard to claim 11, Iwamura in view of Prasad further in view of Smith discloses 
an invention similar to that which is claimed in claim 11. See paper 4, 102 rejection of 
claim 9 and the 103 rejection of claim 9 in this paper for similarities. Iwamura does not 
disclose that the reference point (hand) is characterized by size and location of a 
centroid of the reference point (hand) in each image. 

Jeannin discloses in column 8, lines 30-34 that, "Each object's centroid provides 
a single reference point within each frame that is used to analyze the object's 
movement. The trajectory of the object's centroid is the basis for describing the motion 
or trajectory of the object from frame 201 to frame 204 [see figure 2]." Thus, the 
reference point is characterized by location of a centroid of an object in each image. 
Jeannin further discloses in column 12, lines 57-59, that "z information may be deduced 
from the size variation of the object between consecutive frames." Thus, a reference 
size is inherent, and an object's reference point is characterized by size of the object. 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to modify the invention of Iwamura by having the reference point 
characterized by hand size and a location of a centroid of the hand in each image. One 
would have been motivated to make such a change based on the teaching of Jeannin to 
use an object's centroid as a single reference point to analyze the object's movement 
within each frame and to use change in the object's size to determine z information size. 
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21. In regard to claim 12, Iwamura in view of Prasad further in view of Smith discloses 
an invention similar to that which is disclosed in claim 12. See paper 4 102 rejection of 
claim 9 and the 103 rejection of claim 9 in this paper for similarities. Iwamura does not 
disclose that the first translation is one of a forward and a backward translation, wherein 
the first translation is characterized by a large change in hand size and a relatively small 
change in a centroid of the hand. 

Jeannin discloses in column 12, lines 52-54, that, "The speed or velocity of the 
object for each time instant may be calculated as the local first order derivative of the 
centroid position, normalized by the image size." Also, see 103 rejection of claim 11 
disclosing that Jeannin uses object size and centroid location to classify trajectory 
information, in the z direction in particular. Thus, any forward and backward translation 
(z direction translation) is characterized by object size and centroid location. It is 
inherent that for z direction translation, this would be a large change in hand size and a 
relatively small change in centroid location of the object. This could be the first or any 
translation in the object's trajectory. 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to modify the invention of Iwamura by having the first translation be 
one of a forward and a backward translation, wherein the first translation is 
characterized by a large change in hand size and a relatively small change in a centroid 
of the hand. One would have been motivated to make such a change based on the 
teaching of Jeannin to use the location of an object's centroid as a reference and to 
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deduce z information from an object's size. Thus, the first or any translation in a z 
(forward or backward) direction could be identified. 

22. Claims 26 and 34 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Qiao 6,075,895 in view of Funayama et al. 6,332,038 B1. 

23. In regard to claims 26 and 34, see Paper No. 6 103 rejection of claim 1 . 

24. Claims 27 and 33 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Qiao et al. 6,075,895 in view of Marrin et al. 5,875,257. 

25. In regard to claim 27, Qiao discloses an invention similar to that which is disclosed 
in claim 27. See rejection of claim 1 for similarities. Qiao does not disclose that 
executing the command further comprises the steps of determining the duration of the 
gesture and correlating the duration of the gesture to a intensity and scale in which the 
command is executed. 

Marrin discloses an invention in which the duration of the gesture is determined 
and correlates to a intensity and scale in which the command is executed. See column 
8, lines 25-28 and 43-46, disclosing, "The tempo dictates the speed at which the 
musical composition is rendered, as well as its timing pattern, and is determined 
primarily from two-dimensional gesture, baton speed (or velocity), and the time between 
beats." Thus, the command is the tempo and the duration of the gesture, which 
correlates in scale and intensity to the speed of the baton movement (since speed takes 
into account the amount of motion in a period of time). Further see column 8, lines 43- 
46, disclosing, "Interpreter 235 may be programmed to detect indication of impending 
tempo increases or decreases (e.g., through analysis of baton speed and direction)". 
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Marrin further teaches that his invention "relates to electronic control, and in 
particular to gestural control of systems capable of exhibiting continuous, dynamically 
changeable behavior." 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to modify the invention of Qiao by having executing the command 
further comprise the steps of determining the duration of the gesture and correlating the 
duration of the gesture to a intensity and scale in which the command is executed, as in 
the invention of Marrin. One would have been motivated to make such a change, since 
both inventions "relate to electronic control, and in particular to gestural control of 
systems capable of exhibiting continuous, dynamically changeable behavior." 

26. In regard to claim 33, Qiao further discloses that repetition of a command increases 
the intensity of the command response. It is understood that repetition of a command 
would cause the command to be carried out as many times as the repetition occurs. 
Thus, the intensity of the command response increases accordingly. For example, if a 
command is repeated once, then it is carried out twice and the intensity of the command 
is twice what it would be if not repeated at all. 

27. Claims 28, 29 and 31 rejected under 35 U.S.C. 103(a) as being unpatentable over 
Qiao et al. 6,075,895 in view of Natsuko et al. 6,252,599 B1 . 

28. In regard to claim 28, Qiao discloses an invention similar to that which is claimed in 
claim 28 of the immediate application. See rejection of claim 1 for similarities. Qiao 
does not disclose that the device is a virtual endoscope. 
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Natsuko discloses a virtual endoscope, operated by a user. See column 6, lines 
24-29, disclosing, "The operator determines a viewpoint location (a virtual endoscope 
lens position) in the internal space of the internal body tissue and inputs the viewpoint 
location via the input device 4. A FOV centerline direction (an optical axis direction of a 
virtual endoscope lens) is also input via the input device 4." Natsuko further disclosing 
various input devices that can be used as the input device 4, including a keyboard and 
track ball. See column 13, lines 6-18. 

Qiao further teaches in column 1, lines 10-23, "In many action-oriented electronic 
games, a player guides... with an input device, such as a low-key-count keyboard, a 
joystick or an electronic pointing device, like a mouse... Though one can learn to 
manipulate such input devices, it is not natural to act through a joystick or a keyboard. 
An alternative method to control... through gesture recognition. In such a method, the 
player's gesture controls the character's motion." Note that Qiao also discloses in 
column 12, line 30, "The present invention is not limited to playing games." 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the inventions of Qiao and Natsuko, having a virtual 
endoscope that is gesture-controlled. One would have been motivated to make such a 
change based on the teaching of Qiao that input devices such as keyboards and 
joysticks are not as natural to act through as gesture recognition, and since the 
invention of Natsuko can incorporate various input devices. Further, Qiao teaches that 
his invention can be used for applications other than games. 
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29. In regard to claim 29, see rejection of claim 28. Natsuko further discloses a 
command that corresponds to rotation of the virtual endoscope. See column 1 1 , lines 
50-57, disclosing, "a rotational angle is input via the input device 4. (The rotational 
angle represents a rotational angle around a pivot of the endoscope." 

30. In regard to claim 31 , see rejection of claim 28. Natsuko further discloses a 
command that corresponds to moving the virtual endoscope in a right to left direction. 
Note in the rejection of claim 28, it is disclosed that the user determines a viewpoint 
location and FOV (field of view) centerline location. Thus, the virtual endoscope can 
inherently move in a right to left direction if the user commands a viewpoint or FOV 
centerline location that is to the left of the current location. 

31 . Claims 30 and 32 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Qiao et al. 6,075,895 in view of Natsuko et al. 6,252,599 B1 as applied to claims 28, 29 
and 31 above, and further in view of Iwamura 6,501 ,51 5 B1 . 

32. In regard to claims 30 and 32, Qiao in view of Natsuko disclose an invention similar 
to that which is claimed in claims 30 and 32. See rejections of claims 28, 29 and 31 for 
similarities. Qiao in view of Natsuko do not disclose that the gesture is specifically a 
rotation of a user's hand or a waving of a user's hand from right to left. 

Iwamura discloses an invention in which the rotation of a user's hand and a 
waving of a user's hand from right to left correspond to gesture commands. See figure 
2, depicting the rotation of a user's hand and figures 1 1 and 16, depicting waving of a 
user's hand from upper right to lower left and from lower left to upper right. 
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Iwamure further teaches in column 5, lines 29-52, "The special hand motion is 
not limited to a circular move. Any other special gesture will do... For example, as a 
variation of the circular hand motion, the user 18 may move the hand 20 several times 
(for example twice) toward diagonal direction, for example, lower left to upper 
right... Compared with the circular motion shown in FIG. 2, this is an easier motion for 
the user 18 to make and also easier to detect for the system. A drawback is that such a 
motion is more likely to occur unintentionally than the circular motion and, thus, 
misdetection could occur more frequently... It is a tradeoff." 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to modify the invention of Qiao in view of Natsuko by having 
gestures be a rotation of a user's hand or a waving of a user's hand from right to left, as 
in the invention of Iwamura. One would have been motivated to make such a change 
based on the teaching of Iwamura that "any other special gesture will do" and that the 
selection of the particular gesture for a command is a tradeoff, which is also common 
and conventional knowledge in the selection of gestures for gesture-recognition 
technologies. 

Response to Arguments 

33. Applicant's arguments filed 22 March 4004 have been fully considered but they are 
not persuasive. 

34. In regard to applicant's arguments on pages 8-9 of Paper No. 7, applicant argues 
that Qiao's invention discloses a method in which the action performed by the device is 
a replication of the detected gesture and that in the present invention, a gesture is 



Application/Control Number: 10/001,553 Page 15 

Art Unit: 2673 

detected and it is determined if it corresponds to a valid command. However, in the 
invention of Qiao, a gesture is determined and it is determined if the gesture 
corresponds to a valid command. See previous rejections of claims 1 and 18. Further, 
having the command be to duplicate the detected gesture does not change the fact that 
the gesture is detected and that this command is then executed by the device after 
determining whether it is valid. 

Conclusion 

35. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

Freeman 6,002,808 discloses a system controlled by hand gestures. 

Fujioka 6,674,424 B1 discloses an invention in which an object's motion is used 
to input information. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Laurel E LeFlore whose telephone number is (703) 305- 
8627. The examiner can normally be reached on Monday-Friday 8-4:30. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Joseph Mancuso can be reached on (703) 305-3885. The fax phone 
number for the organization where this application or proceeding is assigned is 703- 
872-9306. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 



LEL 



26 April 2004 



